Skip to content

refactor: rework JsonPatch collection-exclusion filter to match FieldFilterService pattern [DHIS2-21852]#24541

Merged
netroms merged 1 commit into
fix-userrole-jsonpatch-lazy-membersfrom
fix/DHIS2-21852-jsonpatch-collection-filter-review
Jul 23, 2026
Merged

refactor: rework JsonPatch collection-exclusion filter to match FieldFilterService pattern [DHIS2-21852]#24541
netroms merged 1 commit into
fix-userrole-jsonpatch-lazy-membersfrom
fix/DHIS2-21852-jsonpatch-collection-filter-review

Conversation

@jason-p-pickering

Copy link
Copy Markdown
Contributor

Summary

Addresses Jan's review comment on #24489. Targets that PR's branch, not master -- this is a review-feedback follow-up, not a new independent change.

Jan's complaint: the collection-exclusion mechanism in JsonPatchManager.apply() wired its own bespoke Jackson filter (mixin + filter provider, both rebuilt per call) rather than following any existing convention -- "way too complicated and entangled to be flexible or extendable."

This reworks the wiring to mirror the codebase's existing, established pattern for the same problem (skip a getter during Jackson serialization based on per-call criteria, without invoking it): org.hisp.dhis.fieldfiltering.FieldFilterService/FieldFilterMixin, already used for ?fields=.

  • New JsonPatchFilterMixin (@JsonFilter) and JsonPatchExcludedPropertyFilter (SimpleBeanPropertyFilter), shaped like FieldFilterMixin/FieldFilterSimpleBeanPropertyFilter.
  • JsonPatchManager caches one mixin-bound ObjectMapper copy per entity type (ConcurrentHashMap<Class<?>, ObjectMapper>, built lazily via computeIfAbsent) instead of rebuilding the binding on every apply() call. findExcludableNonOwnerCollections (the actual exclusion rule) is unchanged -- it was already schema-driven and generic, not UserRole-specific.
  • The mixin is scoped per-realClass, not bound to Object.class globally. FieldFilterMixin's own filter is path-aware, so a global binding is safe for it; ours matches by property name only. A global binding was tried first here and found, in review, to silently strip Sharing.users/Sharing.userGroups (present on every IdentifiableObject) whenever a same-named top-level collection was excluded -- e.g. a scalar UserRole PATCH would wipe sharing.users. Caught before merge, never released. Scoping per-class makes that cross-type collision structurally impossible. Regression test: testUserRoleSharingUsersSurviveScalarPatch.
  • Answers Jan's second question (should an explicit patch to an excluded property throw?): no -- findExcludableNonOwnerCollections already un-excludes a property referenced by a patch path, so it falls back to full (slower, correct) hydration rather than being silently dropped. Replaced the previous testUserRoleUsersPathPatchDoesNotThrow (asserted only assertNotNull, would pass either way) with testUserRoleUsersPathPatchFallsBackToFullHydration, which asserts Hibernate.isInitialized(...) actually flips to true, mutation-tested against the fallback logic.

No functional change to PATCH behavior beyond fixing the Object.class regression introduced and caught within this same review cycle.

Separately (not in this PR): David asked whether PUT has the same issue -- confirmed it doesn't (AbstractCrudController.putJsonObject parses the request body into a fresh transient object, never round-trips the persisted entity through Jackson). A dedicated PUT perf test to prove that under real load is tracked as a follow-up.

Test plan

  • mvn spotless:apply clean
  • JsonPatchExcludedPropertyFilterTest (new, unit): 2/2 passing
  • JsonPatchManagerTest (dhis-test-integration): 14/14 passing (13 original + testUserRoleSharingUsersSurviveScalarPatch)
  • UserControllerTest (dhis-test-web-api): 52/52 passing, unaffected
  • dhis-test-performance module: mvn test-compile passes (Gatling UserRolesPerformanceTest untouched)
  • Mutation-tested: the Object.class collision bug, the fallback-not-silently-dropped behavior, and the filter-omission mechanism itself were each confirmed to fail for the right reason when deliberately broken, then reverted

AI Assisted

🤖 Generated with Claude Code

@jason-p-pickering
jason-p-pickering force-pushed the fix/DHIS2-21852-jsonpatch-collection-filter-review branch 2 times, most recently from 06884f3 to 81df701 Compare July 23, 2026 14:41
@jason-p-pickering

Copy link
Copy Markdown
Contributor Author

Local performance + Glowroot verification

Ran UserRolesPerformanceTest locally against a live server built from this branch (81df701088, reconciled with Morten's latest fix-userrole-jsonpatch-lazy-members commits), targeting role MoRvPzDH7lc (83k members).

Gatling results — 132/132 requests OK, 0 KO:

Request n p50 p95 max
PATCH scalar (empty role) 20 44ms 57ms 67ms
PATCH scalar (large role, 83k members) 30 33ms 39ms 41ms
GET narrow fields (large role) 70 6ms 9ms 10ms

Large-role PATCH latency is in the same band as the empty-role control (actually slightly faster here, just run-order noise) — the membership-size-independence the fix is meant to guarantee.

Glowroot trace on a single large-role PATCH (/api/44/userRoles/MoRvPzDH7lc, 183ms total): 21 SQL queries captured. 20 of them carry DHIS2's /* controller='UserRoleController',method='patchObject' */ SQL comment tag, i.e. issued directly by the synchronous PATCH-handling code — all lightweight, single-row lookups (schema/category/attribute checks, org-unit checks, the UPDATE userrole, one authorities INSERT). None of them reference userrolemembers.

The one query that does join userrolemembers has no patchObject tag, runs on Glowroot's auxThreadRootTimer (not the main request thread), and is still "active": true at trace-capture time (i.e. still running after the response had already been sent). That's the separate, already-shipped async session-invalidation feature (04af5ab35c, DHIS2-21854) firing off-thread by design — not the old eager-hydration bug resurfacing.

Net: the actual PATCH logic this PR touches never hydrates userrolemembers, confirmed live on real membership size, not just in unit/integration tests.

…FilterService pattern [DHIS2-21852]

Addresses review feedback on #24489 (Jan Bernitt): the per-call `.addMixIn(realClass, ...)` +
inline mixin + filter-provider wiring in JsonPatchManager.apply() was rebuilt from scratch on every
call and didn't follow any existing convention. Reworks it to mirror the codebase's established
pattern for the same underlying problem -- skipping a getter during Jackson serialization based on
per-call criteria, without invoking it -- already used for the `?fields=` GET path
(org.hisp.dhis.fieldfiltering.FieldFilterService/FieldFilterMixin).

- New `JsonPatchFilterMixin` (`@JsonFilter`) and `JsonPatchExcludedPropertyFilter`
  (`SimpleBeanPropertyFilter`), shaped like `FieldFilterMixin`/`FieldFilterSimpleBeanPropertyFilter`.
- `JsonPatchManager` caches one mixin-bound `ObjectMapper` copy per entity type
  (`ConcurrentHashMap<Class<?>, ObjectMapper>`, built lazily via `computeIfAbsent`) instead of
  rebuilding the mixin binding on every `apply()` call. `findExcludableNonOwnerCollections` (the
  exclusion rule itself) is unchanged -- it was already schema-driven and generic.
- The mixin is scoped per-`realClass`, not bound to `Object.class` globally: `FieldFilterMixin`'s own
  filter is path-aware, so a global binding is safe for it, but `JsonPatchExcludedPropertyFilter`
  matches by property name only. A global binding was tried first and found, in review, to silently
  strip `Sharing.users`/`Sharing.userGroups` (present on every `IdentifiableObject`) whenever a
  same-named top-level collection was excluded, e.g. on a scalar UserRole PATCH. Scoping per-class
  makes that cross-type collision impossible. Regression test:
  `testUserRoleSharingUsersSurviveScalarPatch`.
- Answers Jan's second question (should an explicit patch to an excluded property throw?): no --
  `findExcludableNonOwnerCollections` already un-excludes a property referenced by a patch path, so
  it falls back to full (slower, correct) hydration rather than being silently dropped. Replaced the
  previous `testUserRoleUsersPathPatchDoesNotThrow` (asserted only `assertNotNull`, would pass either
  way) with `testUserRoleUsersPathPatchFallsBackToFullHydration`, which asserts
  `Hibernate.isInitialized(...)` actually flips to `true`, mutation-tested against the fallback logic.

No functional change to PATCH behavior beyond fixing the Object.class regression introduced and
caught within this same review cycle (never released). All existing JsonPatchManagerTest coverage
(14 tests) plus the new JsonPatchExcludedPropertyFilterTest (2 tests) pass; UserControllerTest (52
tests, dhis-test-web-api) unaffected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@jason-p-pickering
jason-p-pickering force-pushed the fix/DHIS2-21852-jsonpatch-collection-filter-review branch from 81df701 to 975d100 Compare July 23, 2026 15:25
@sonarqubecloud

Copy link
Copy Markdown

@netroms
netroms merged commit 975d100 into master Jul 23, 2026
24 of 25 checks passed
@netroms
netroms deleted the fix/DHIS2-21852-jsonpatch-collection-filter-review branch July 23, 2026 16:09
@netroms

netroms commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

FF-merged into parent PR #24489 (fix-userrole-jsonpatch-lazy-members @ 975d100fb8).

This stacked refactor is now part of the main DHIS2-21852 branch. Closing as incorporated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants